Ordering as privileged information

We propose to accelerate the rate of convergence of the pattern recognition task by directly minimizing the variance diameters of certain hypothesis spaces, which are critical quantities in fast-convergence results. We show that the variance diameters can be controlled by dividing hypothesis spaces into metric balls based on a new order metric. This order metric can be minimized as an ordinal regression problem, leading to a LUPI application where we take the privileged information as some desired ordering, and construct a faster-converging hypothesis space by empirically restricting some larger hypothesis space according to that ordering. We give a risk analysis of the approach. We discuss the difficulties with model selection and give an innovative technique for selecting multiple model parameters. Finally, we provide some data experiments.